A multidimensional generalized many-facet Rasch model for rubric-based performance assessment
نویسندگان
چکیده
Abstract Performance assessment, in which human raters assess examinee performance a practical task, often involves the use of scoring rubric consisting multiple evaluation items to increase objectivity evaluation. However, even when using rubric, assigned scores are known depend on characteristics rubric’s and raters, thus decreasing ability measurement accuracy. To resolve this problem, item response theory (IRT) models that can estimate while considering effects these have been proposed. These IRT assume unidimensionality, meaning measures one latent ability. In practice, however, assumption might not be satisfied because designed measure sub-abilities constitute targeted address issue, study proposes multidimensional model for rubric-based assessment. Specifically, proposed is formulated as extension generalized many-facet Rasch model. Moreover, No-U-Turn variant Hamiltonian Markov chain Monte Carlo algorithm adopted parameter estimation method The useful only improving accuracy, but also detailed analysis quality construct validity. demonstrates effectiveness through simulation experiments application real data.
منابع مشابه
Many-Facet Rasch Measurement
This chapter provides an introductory overview of many-facet Rasch measurement (MFRM). Broadly speaking, MFRM refers to a class of measurement models that extend the basic Rasch model by incorporating more variables (or facets) than the two that are typically included in a test (i.e., examinees and items), such as raters, scoring criteria, and tasks. Throughout the chapter, a sample of rating d...
متن کاملRater Errors among Peer-Assessors: Applying the Many-Facet Rasch Measurement Model
In this study, the researcher used the many-facet Rasch measurement model (MFRM) to detect two pervasive rater errors among peer-assessors rating EFL essays. The researcher also compared the ratings of peer-assessors to those of teacher assessors to gain a clearer understanding of the ratings of peer-assessors. To that end, the researcher used a fully crossed design in which all peer-assessors ...
متن کاملA Many-facet Rasch Model to Detect Halo Effect in Three Types of Raters
Raters play a central role in rater-mediated assessment, and rater variability manifested in various forms including rater errors contributes to construct-irrelevant variance which can adversely affect an examinee’s test score. Halo effect as a subcomponent of rater errors is one of the most pervasive errors which, if not detected, can result in obscuring an examinee’s score and threatening val...
متن کاملIntellectual Achievement Responsibility. a Many-facet Rasch Model Analysis of the Scale by Crandall, Katkovsky and Crandall
The present study consists in the analysis of the responses of 485 high school Italian students to the items of the Intellectual Achievement Responsibility questionnaire (IAR; Crandall, Katkovsky, & Crandall, 1965) in view of the definition of a new form of the scale. The Italian version by Nigro and Galli (1988) was applied to the participants. The data were analyzed by means of the Many-Facet...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: Behaviormetrika
سال: 2021
ISSN: ['0385-7417', '1349-6964']
DOI: https://doi.org/10.1007/s41237-021-00144-w